January 25, 1974 


MEMORANDUM TO TOBACCO INDUSTRY FILE 

The following memorandum records what transpired 
at the meeting between industry and Federal Trade Commission 
representatives at the Commission on January 18, 1974, re¬ 
lating to the Commission's "tar" and nicotine testing. 
Present for the Commission were: Harold Pillsbury, director 
of the Commission's laboratory; James O'Connor, Commission 
scientific director; Charles Marx, statistician; ana Russ 
Hatchl and Richard Herzog, Commission attorneys. Present 
for the industry were: Horace Kornegay, Dr. Alex Spears, 

Dr. William Steele, Dr. Herbert Arkin, HTA and AJT. 

Mr. Kornegay began the meeting by stating that 
the industry appreciated the opportunity to discuss aspects 
of the Commission's testing which relate to the integrity 
of the Commission's results and are therefore of importance 
to the Commission and to the public as well as to the in¬ 
dustry . 

HTA stated that the accuracy of the Commission's 
results was of particular importance because of the indus¬ 
try's voluntary agreement to publish the Commission's "tar" 
and nicotine figures in cigarette advertising. 

AJT introduced Dr. A.rkin and stated that Dr. 

Arkin would summarize his analysis of the Commission's 
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last round of testing and that copies of this analysis 
and the one prior to it would be made available to the 
Commission at the conclusion of the meetiny. 

Dr, Arkin then summarised his analysis of the 
latest round of Commission testing data. He began by 
noting that calculational errors had increased in the 
last round of testing and that many figures were illegible. 
Dr. Arkin distributed a copy of his summary of calcula¬ 
tional errors, which is attached as Exhibit 1 hereto. 

Dr. Arkin next stated that he has been very dis¬ 
turbed by the large number of values obtained in testing 
which have been deleted in the Commission's calculations. 

He stated that proper use of any technique for excluding 
outliers permitted the discarding of only a small number 
of values and that better technique required that values 
only be discarded where there was some assignable cause 
for the high or low value. A summary of deletions was 
distributed by Dr. Arkin, and a copy is attached as Exhibit 
2 hereto. Dr. Arkin then stated that in addition to these 
deletions many other values were excluded by the Commission 
by a process of pasting new results over old ones on the 
Commission's laboratory sheets. He observed that in his 
view there is no justification for this practice. 
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Dr* Arkin next discussed variations in the level 
of the tests performed by the Commission as reflected by 
shifts in monitor values either for one day or a whole 
series of days during a single test. These shifts were 

accompanied by a corresponding shift in a majority of the 

brands smoked on those days. Dr. Arkin suggested that 
this problem could be solved if the Commission maintained 
process control charts which would indicate when there had 
been a deviation and would enable the Commission to stop 
testing and correct the source of the deviation. 

Mr. Herzog asked if it were possible that the 
defect were in the monitor and that monitor values would 
shift comparably in all laboratories at the same time. 

Dr. Arkin stated that this was extremely unlikely. 

Mr. Hatchl asked how great the shifts in the 

monitors were. 

Dr. Arkin replied that these shifts in monitors 
were as much as 0.9 mgm. of "tar." 

Mr. Herzog asked what the TITL's policy was on 
discarding data. 

Dr. Steele stated that he discarded values only 
where there was some assignable cause. 

Dr. Arkin then discussed the double rounding 
which had been done in the last round of Commission results 
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and the inconsistencies which this had produced in the 
Commission’s results. 

Mr. Pillsbury stated that this problem had been 
caused by the computer, that it had been solved, and that 
it would not occur again. Mr. Pillsbury also stated that 
a statement would be included in the next Commission re¬ 
port of test results explaining the policy on rounding but 
that the basic format for presentation of results would be 
retained. 

Mr. I-Iatchl asked what sources of experimental 
error could give rise to changes which would be reflected 
by the control chart. 

Dr. Spears explained some of the sources of 
error, including humidity, variations in smoking equipment, 
variations in the spectrophotometer, and temperature vari¬ 
ations . 

AJT illustrated a process control chart which 
showed experimental deviations. AJT then asked Pillsbury 
the reason for the Commission 1 s pasting over of results. 

Mr. Pillsbury stated that if the monitor values 
for any day fell outside of certain limits, then the entire 
run would be discarded and a new column would be pasted 
over the discarded data on the card used to record data. 
This new column would be used to record data from a resmok¬ 
ing of this run. Mr. Pillsbury stated that the deletions 
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(as contrasted with the paste-overs) were done by follow¬ 
ing the Dixon and Massey outlier test and that the Commis¬ 
sion laboratory had rigid rules for deleting data. A copy 
of the Commission's rules for deleting data was distributed 
and is attached as Exhibit 3 hereto. 

Dr. Arkin questioned the use of the Dixon and 
Massey outlier technique when dealing with testing of a 
product with inherent variability such as cigarettes. 

Mr. Marx stated that the Commission was in a 
quandary because it had much data, some of which was 
spurious and might prejudice the results. 

HTA stated that it was impossible to determine 
which data was spurious. 

AJT stated that quite apart from the problem of 
deleting outliers, there was no justification for pasting 
over data. The original data, even if excluded from the 
calculations, should be retained to permit defects in the 
Commission's testing to be found. 

Mr. Herzog asked why values which exceeded the 
average by certain limits should be retained. 

AJT stated that these values might result from 
the different storage conditions to which cigarettes are 
exposed and should be included if the tests are 'to repre¬ 
sent cigarettes available to consumers. AJT also stated 
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that inclusion of these results might not change the mean 
values for brands, but might affect only the precision of 
the results. 

Dr. Spears stated that he would not be concerned 
if the Commission used an outlier technique to discard a 
very small percentage of data, but that it was improper 
to discard a large percentage of all data as the Commis¬ 
sion did. Dr. Spears stated that comparison of FTC and 
TITL results on the last test by means of regression anal¬ 
ysis showed reasonably good agreement. On the prior test 
agreement had been poor because the Commission lacked 
sufficient controls to detect an experimental shift in an 
experimental condition such as humidity. 

Dr. Arkin distributed his summary of Comrnission 
paste-overs, which is attached as Exhibit 4 hereto. 

AJT asked why the last round of testing showed 
almost no paste-overs for monitors, but many for brand 
values. 

Mr. Pillsbury stated that monitor values accom¬ 
panying brand paste-overs had not been pasted over, but they 
had not been included in the calculations. 

Dr. Arkin suggested that instead of pasting over 
data, the Commission, if it was determined to exclude 
whole runs, should at least keep the data and stamp it 
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discarded. This would distinguish this data from other- 
data which is excluded by the outlier technique and stamped 
deleted. 

HTA stated that the industry had reserved and 
continues to reserve the right to object to any aspects of 
the Commission's methodology in the future or to propose 
any changes. 

Mr. Kornegay suggested that the Commission con¬ 
sider publishing its results twice a year on regular dates 
.to facilitate use of these figures in advertising. 

Mr. Hatchl stated that this might not always be 

possible. 

Mr. Kornegay suggested as an alternative that 
the industry be given a reasonable amount of advance notice 
of when the results would be likely to appear. 

AJT distributed copies of Dr. Arkin's analyses 
of the last two Commission tests. AJT stated that this 
industry group would be available for further consultation 
on the subjects covered at this session if that was de¬ 
sired by the Commission and that the industry might want 
to suggest another meeting in the future to discuss the 
subjects covered at this meeting as well as related sub- 
j ects. 

Allan J. Topol 


Enclosures 
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